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The  Joint  Center  for  Robotics  (JCR)  Virtual  Systems  Integration  Lab  (VSIL)  is  a  combination  of  Robotics 
Software  Models  and  Tools  used  to  stimulate  Hardware  or  perform  evaluations  on  developing  concepts.  The 
models  have  been  developed  at  the  Tank  Automotive  Research,  Development  Engineering  Center  (TARDEC)  or 
other  RDECOM  labs  and  centers.  JCR  VSIL  is  focusing  on  supporting  the  Robotic  Systems  Joint  Project  Office  (RS 
JPO)  in  testing  of  their  developing  Interoperability  Profiles  for  FYIO. 

The  RS  JPO  Interoperability  Profiles  will  need  a  facility  for  testing  compliance  to  the  profile  attributes.  The 
JCR  VSIL  is  building  the  testing  compliance  facility  in  coordination  with  the  development  of  the  Profiles.  The  first 
demonstration  is  planned  for  May,  2010.  This  demonstration  will  measure  the  latency  of  controller(s)  to  simulated 
and  ‘live’  robotics  platform(s)  using  the  RS  JPO  Interoperability  Profile  using  JAUS  AS-4  message  standard. 
Simulated  SUGV  will  respond  to  mobility,  manipulator  and  pose  control  operational  messages.  A  message  analyzer 
will  verify  the  correct  message  is  being  passed.  A  simulated  TALON  will  also  be  measured  through  a  series  of 
JAUS  AS-4  messages.  ‘Live’  hardware  will  consist  of  TARDEC  Intelligent  Ground  Systems  (IGS)  Robotics 
Autonomous  Mobility  Platform  (RAMP)  pan/tilt  sensor  and  operator  sensor  (Software  surrogate  may  be  just 
prototype  discovery  /  authentication  services).  The  Interoperability  Profile  will  be  executed  using  two  different 
OCUs  to  manipulate  the  virtual  and  ‘live’  hardware. 

This  paper  will  discuss  the  tradeoffs,  architecture  and  design  of  the  testing  system  and  hardware  in  support  of 
the  May  demonstration. 


Introduction 

The  Robotics  Systems  Joint  Project  Office  (RS 
JPO)  has  launched  an  initiative  to  identify/define 
interoperability  standards  to  be  organized  and 
maintained  within  a  UGV  Interoperability  Profile 
(IOP)  such  that  they  can  be  employed  by  robotics 
Program  Managers  in  the  acquisition  of  future  ground 
robotics  system  programs  of  record,  the  upgrade  of 
currently  fielded  systems,  and  the  evaluation  and 
acquisition  of  COTS  components. 

A  primary  goal  of  this  initiative  is  to  leverage 
existing  and  emerging  standards,  to  the  extent 
possible,  within  the  UxV  community  such  as  private 
sector  groups,  Joint  Architecture  for  Unmanned 
Systems  (JAUS)  and  the  Army  Unmanned  Aircraft 
Systems  (UAS)  Project  Office  Interoperability  Profile 
with  an  end  goal  of: 

•  Facilitating  interoperability  among  new  UGV 

initiatives  and  legacy  systems; 


•  Facilitating  interoperability  between  controller 
and  UxV  robotic  system(s); 

•  Facilitating  collaboration  between  UGV  and 

UAS  systems; 

•  Providing  a  path  forward  to  standardize 
interoperable  technology  solutions  to  minimize 
current  force  UGV  gaps;  and 

•  Promoting  payload  and  on-board  subsystem 

commonality/interchange  across  fielded  UGV 

systems. 

As  part  of  the  process  for  developing  the  IOP 
Profile,  Requests  For  Information  (RFI)  were  sent  out 
to  industry.  Through  the  RFI  process  it  was  shown 
that  industry  believes  that  there  needs  to  be  an 
independent  testing  and  compliance  facility  for 
analyzing  new  technology  and  platforms.  In 

addition,  there  is  a  need  for  certifying  IOP  messaging 
compliance  to  make  sure  the  new  technology  and 
platforms  can  communicate  correctly.  Further,  there 
is  a  need  for  standardized  test  procedures  that  ensures 
that  contractors  in  the  industry  can  develop  to  and 
meet  the  goals  of  the  IOP  Profile.  The  purpose  of  the 
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Virtual  Systems  Integration  Lab  (VSIL)  is  to  provide 
methods  of  verification  and  validation  to  meet  these 
needs. 

A  demonstration  system  was  developed  to  show 
initial  latency  testing  capabilities,  monitoring  and 
validation  of  JAUS  messages  sent  over  the  network, 
and  introduction  of  some  of  the  IOP  ooncepts 
developed.  The  system  was  comprised  of  two 
simulated  SUGV  models,  an  Operational  Control 
Unit,  and  a  surrogate  UGV  platform  with  pan-tilt- 
zoom  (PTZ)  camera.  The  system  is  depicted  in 
Figure  1. 


Surrogate  RAMP 
Platform  with  real  pan  / 
tilt  room  sensor 


Figure  1:  Overall  System 

Demonstration  System 

The  demonstration  system  leveraged  three 
products  from  TARDEC  Intelligent  Ground  Systems: 
the  Embedded  Simulation  System  (ESS)  with  two 
simulated  SUGV  assets,  the  Scalable  Soldier 
Machine  Interface  (SSMI)  software  with  layout  based 
on  a  version  used  for  the  Common  Controller 
Interoperability  Networking  Evaluations,  and  a 
surrogate  RAMP  system  utilizing  a[l]  Pan  Tilt  Zoom 
(PTZ)  camera.  The  communication  network  was 
comprised  of  two  gigabit  switches  and  a  bridge  from 
one  switch  to  the  other. 

The  system  uses  SAE  JAUS  messages  to 
communicate  from  OCU  to  the  simulated  assets  and 
from  the  OCU  to  the  PTZ  camera.  The  PTZ  camera 
also  had  a  simple  OCU  developed,  that  also  used 
JAUS.  The  purpose  behind  the  two  OCUs  was  to 
show  that  through  the  IOP  rules,  both  OCUs  could 
operate  a  common  piece  of  hardware  or  software 
utilizing  the  JAUS  message  set  and  the  IOP  rules. 

JAUS  Message  Analyzer 


SAE  JAUS  4.0  messages  are  used  for 
communications  between  subsystems  of  the 
demonstration  system.  In  addition  to  the  standard  the 
IOP  Profile  has  rules  to  disambiguate  and  extend  the 
message  set,  promoting  interoperability.  The 
message  analyzer  was  developed  to  examine  network 
traffic,  identifying  JAUS  messages  and  their 
compliance  to  the  standard  and  IOP  Profile.  It  can 
also  help  to  verify  correct  sequencing  of  messages. 


Figure  2:  Message  Analyzer  with  Custom  Plug-in 

The  message  analyzer  is  a  [3]  plug-in  (Figure  2).  The 
analyzer  has  the  ability  to  capture  and  display 
network  traffic.  The  plug-in  extends  its  capabilities 
to  include  analysis  of  JAUS  messages,  and  sequences 
of  messages.  Figure  (3)  shows  where  in  the  network 
this  tool  was  used. 


Message  Analyzer 


Test  Article 
OCU 


Figure  3:  Message  Analyzer  Use 

Each  JAUS  message  is  comprised  of  a  JAUS  header 
followed  by  its  corresponding  fields.  All  of  which  is 
serialized  into  a  non-human  readable  format. 

The  analyzer  filters  the  network  traffic  looking 
for  the  embedded  header  in  order  to  identify  the 
message  as  a  JAUS  message.  It  then  parses  each  of 
the  fields  creating  a  human  readable  format  which  is 
then  displayed  in  the  plug-in.  The  plug-in  maintains 
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the  order  in  which  the  messages  were  sent.  This 
leads  to  validation  of  the  message  format,  as  well  as, 
the  sequence  in  which  they  are  sent.  The  capture  of 
message  sequences  enables  verification  of  required 
exchanges  to  accomplish  functions  such  as  the 
discovery,  and  login.  Future  work  will  include  the 
ability  to  monitor  sequences  via  a  specified,  user- 
input,  sequence  diagram.  In  other  words,  the  user 
will  create  a  sequence  diagram  that  details  what 
messages  and/or  data  is  to  be  exchanged,  and  the 
analyzer  will  look  for,  and  validate,  that  exchange. 

Latency 


problems  with  this  type  of  latency  measurement.  The 
first  is  that  the  generation  and  recording  of  the 
timestamp  takes  up  CPU  cycles  and  thus  adds  to  the 
latency.  The  second  is  that  the  timestamps  on 
different  CPUs  have  to  be  based  on  synchronized 
clocks.  The  third  issue  is  that  part  of  the  overall 
latency  cannot  be  measured.  For  instance,  using  this 
type  of  measurement  technique,  the  joystick’s 
reaction  latency  (e.g.  mechanical  rotation 
electrical  signal)  and  the  robotic  platform’s  reaction 
latency  (e.g.  command  motion )  cannot  be 

measured.  Equation  (1)  shows  an  example  latency 
thread. 


Latency  requirements  govern  the  overall  system 
performance  and  can  be  defined  at  various  levels. 
For  example,  the  control  of  a  UGV  manipulator  can 
be  measured  from  input  on  the  OCU  to  the 
commanded  movement  of  the  manipulator  or  at 
various  points  along  the  thread  to  include  OCU  to 
communications  link  transmission,  communications 
link  transmission  to  communications  link  reception, 
communications  link  reception  to  manipulator.  The 
IOP  Profile  will  define  what  latencies  are  of 
importance  and  as  the  profile  is  extended  new  types 
of  tests  will  be  identified  and  implemented. 

Vendors  following  the  IOP  Profile  should  be  able 
to  calculate  latencies  along  critical  paths  for  then- 
systems.  However,  in  order  to  prove  that  the 
vendor’s  platform  meets  the  IOP  Profile,  the  vendor 
might  need  to  make  known  proprietary  information 
used  in  calculating  these  latencies.  The  tests  that  will 
be  discussed  in  the  following  sections  should  lead  to 
a  method  that,  not  only,  protects  a  vendor’s 
intellectual  property,  as  well  as,  make  measurements 
across  differing  platforms  directly  comparable. 

Measurements  of  latency  can  be  grouped  into  two 
kinds:  intrusive,  and  non-intrusive.  Intrusive 
measurements  are  typically  done  with  timestamps 
embedded  into  the  programming  code  at  various 
places  of  interest  within  a  thread.  For  example,  a 
timestamp  might  be  generated  when  a  joystick  is 
pushed  forward  which  issues  a  command  to  a  robotic 
platform  ordering  it  to  move  forward.  On  the 
platform  side,  another  timestamp  is  used  to  capture 
when  the  robotic  platform  has  received  the  command. 
This  latency  gives  the  latency  between  the  generation 
and  consumption  of  the  command.  There  are  a  few 
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In  order  to  combat  these  problems  of  intrusive 
methods,  non-intrusive  methods  have  been  devised. 
An  example  is  the  TARDEC  Robotic  Vehicle  Control 
Architecture  (RVCA)  ATO  where  GPS  and  high 
speed  camera  were  used  to  obtain  tele-operation 
control  and  video  latency  measurements.  Such  a 
method  enables  testing  in  the  field  with  high  degree 
of  accuracy.  However,  such  a  method  can  also  be 
costly  to  implement.  For  this  reason,  the  JCR-VSIL 
team  decided  to  investigate  and  validate  cost 
effective  methods. 

For  this  demonstration,  three  methods  of  non- 
intrusive  methods  of  latency  measurement  were 
introduced.  These  methods  use  Micro  Electro- 
Mechanical  Sensors  (MEMS)  in  order  to  measure 
mechanical  actions  or  electrical  signals  at  the  end¬ 
points  (i.e.  generation  and  consumption  of 
commands/signals)  of  a  latency  pathway.  There  are  a 
variety  of  MEMS  type  sensors  on  the  market  that  can 
be  utilized  for  measuring  latencies.  For  instance, 
through  the  use  of  a  tilt-sensor  (specific  application 
of  an  accelerometer)  or  a  rate  gyro  (measures 
rotation),  the  test  application  can  measure  the 
mechanical  movement  of  a  joystick,  indicating  when 
the  operator  has  generated  a  command  to  the  robotic 
platform. 

The  MEMS  are  placed  at  endpoints  of  a  latency 
path  to  be  measured.  For  instance,  the  latency  from 
the  generation  of  a  command  to  the  consumption  of 
the  command  and  consequent  actuation,  would  have 
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a  MEMS  mounted  on  a  joystick  and  another  MEMS 
mounted  on  the  actuator  the  command  is  sent  to.  The 
joystick  and  the  actuator  are  the  endpoints  of  the 
latency  path.  The  signals  from  each  of  the  MEMS 
are  read  via  an  oscilloscope.  Time  is  derived  from 
the  oscilloscopes  measurements,  and  is  centric  to  the 
system,  i.e.  no  synchronization  across  platforms  is 
necessary.  Typically,  the  measurement  is  taken 
between  the  rising  edges  of  two  analog  pulses 
recorded  from  the  two  sensors,  positioned  at  the  end 
points  of  the  latency  to  measure. 

These  types  of  measurements  do  have  some 
uncertainty  associated  with  them,  such  as  the 
response  time  of  the  sensors,  but  through  careful 
design  and  methodical  sensor  selection,  this  can  be 
minimized  and  characterized.  Also,  measurements 
need  to  take  into  account  human-error.  For  instance, 
a  rate  gyro  attached  to  a  joystick  will  often  have 
“bounces”  in  the  beginning  of  the  rising  edge  of  the 
pulse  due  to  a  human  pressing  forward  at  an 
inconsistent  speed  on  the  joystick.  If  the  falling  edge 
of  the  curve  was  used  to  determine  the  latency  in 
turning  off  the  actuator,  the  joystick  returning  to  its 
central  position,  and  “bouncing”  would  have  to  be 
taken  into  account.  In  order  to  minimize  the 
uncertainty  introduced  by  human  or  equipment 
variability,  an  electro-mechanical  harness  may  be 
devised.  Without  a  mechanism  with  repeatable  force 
in  actuating  control  devices,  it  will  be  difficult  to 
characterize  the  precision  of  the  measurement 
method  (unless  the  human  variability  is  considered 
part  of  the  method).  The  resources  and  time 
allocated  for  this  initial  capability  demonstration  did 
not  allow  for  such  development,  but  this  is  an  aspect 
that  should  be  investigated  in  future  follow-on  effort. 

Non-lntrusive  Latency  Measure  Concepts 

There  were  three  latency  measurement  tests 
developed  for  the  demonstration  system.  The  first 
was  a  round  trip  latency  from  command  and 
control(C2)  teleop  video  feedback  on  the  OCU. 
The  second  was  the  latency  from  C2  actuator 
movement.  The  third  method  was  a  worst  case 
latency  from  video  recording  display  on  an  OCU. 
Each  of  these  is  described  in  more  detail  in  the 
following  sub-sections,  as  well  as,  two  sections 
dedicated  to  the  types  of  sensors  used.  Following  the 


sections  on  each  type  of  latency  measurement  is  a 
corresponding  equation  that  details  the  path  and 
endpoints  of  what  is  being  measured. 

Rate  gyros  measure  rotation  about  an  axis.  A  rate 
gyro  can  have  multiple  axis  of  rotational 
measurement;  however,  a  single  axis  of  rotation  was 
all  that  was  needed  for  this  demonstration  system. 
The  gyros  themselves  have  maximum  frequency 
response  of  100  Hz,  translating  into  roughly  10ms  of 
potential  error  that  needs  to  be  introduced  into  the 
overall  latency  calculation.  For  some  latency 
requirements,  this  may  be  sufficient  accuracy,  but 
many  systems  may  require  sub  millisecond  accuracy. 

In  addition  to  the  rate  gyros,  accelerometers  were 
considered,  specifically  a  tilt  sensor  application  of  an 
accelerometer.  Accelerometers  often  times  have 
quicker  response  times.  This  also  implies  that  they 
are  subject  to  more  noise  introduced  by  the 
environment.  For  this  experiment,  an  accelerometer 
was  not  tried,  but  in  the  future  the  results  from  the 
two  types  of  sensors  should  be  compared  in  order  to 
determine  the  best  testing  equipment. 

The  photo  detector  measures  light  with 
wavelengths  from  350nm  to  1  lOOnm  and  supplies  an 
output  voltage  proportional  to  the  wavelengths  being 
measured.  The  output  voltage  is  based  on  the  gain, 
and  the  loading  of  the  circuit.  The  gain  is  adjustable 
via  an  8  position  switch  with  gains  from  OdB  to 
70dB.  Response  times  of  the  photo-detector  are 
dependent  on  the  gain  setting,  and  the  intensity  of  the 
light  being  measured,  and  therefore  need  to  be 
determined  before  the  latency  measurement  is  taken. 

A  method  of  determining  the  response  time  of  the 
photo-detector  is  to  use  a  function  generator.  By 
connecting  the  function  generator  to  the  oscilloscope 
and  having  a  square  wave  pulse  a  LED,  the  photo¬ 
detector’s  rise  time  can  be  measured.  This  response 
time  does  not  correspond  directly  to  the  response 
time  introduced  by  measuring  light  from  an  LCD 
panel.  This  is  an  ideal  method  of  measuring  the 
response  time  of  the  photo-detector.  A  method  of 
characterizing  the  response  time  from  resultant  from 
a  change  in  light  color  or  intensity  induced  by  an 
LCD  panel,  which  is  a  slower  response,  needs  to  be 
addressed  in  the  future. 
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C2  ->  Teleop  Video  Feedback  on  the  OCU  02  ->  Actuator  Movement 


One  measurement  that  is  of  significant 
importance  to  the  UxV  domain,  is  the  time  it  takes 
from  the  generation  of  a  commanded  movement  to 
when  the  operator  sees  the  resultant  imagery  on  the 
OCU.  It  is  widely  understood  that  successful  tele¬ 
operation  is  dependent  on  the  latency  between  when 
the  operator  commands  a  movement,  and  when  the 
operator  perceives  the  feedback  on  the  video  screen. 
In  order  to  measure  this  latency,  a  rate-gyro  was 
attached  to  the  joystick  to  measure  the  time  at  which 
the  command  is  initiated,  and  a  photo-detector  was 
placed  in  front  of  the  OCU  screen  to  detect  a  change 
from  black  to  white.  This  was  done  with  the 
simulated  assets,  using  a  special  terrain  database.  The 
simulated  database  contained  a  special  mode  where 
the  robotic  platform  with  its  camera  sensor  would  be 
positioned  inside  of  a  virtual  3-dimensional  box. 
This  virtual  box  contained  white  walls,  with  one 
black  wall.  Initially,  the  simulated  asset’s  camera- 
under-test  faces  the  black  wall.  When  the  command 
to  turn  is  initiated  via  the  joystick  the  asset  turns 
within  the  box,  to  face  a  white  wall.  The  resultant 
imagery  displayed  on  the  OCU’s  screen  switches 
from  black  to  white,  which  is  captured  via  the  photo¬ 
detector. 


Electro-Mechanical  Non-lntmslve  Latency  Measurement  -  1 
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Figure  4:  Round  Trip  Latency  C2— >  Video  Feedback 

The  gyro  captures  the  initiation  of  the  command, 
and  the  photo-detector  captures  the  switch  from  black 
to  white,  and  the  difference  between  thase  two 
measured  times  is  the  round  trip  latency. 
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Electro-Mechanical  Non-lntruslve  Latency  Measurement  -  Case  ? 
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Figure  5:  Latency  Measure  C2  — >  Actuator  on  PTZ 

Through  the  use  of  two  rate-gyros  (measure 
rotation  in  V/°/S),  the  latency  from  the  generation  of 
a  commanded  movement  to  when  the  actuator 
responds  and  the  movement  occurs,  can  be  measured. 
The  resulting  latency  would  include  the  j  oystick  and 
platform  motion  reactions,  without  the  use  of 
software  captured  timestamps.  One  rate  gyro  is 
placed  on  the  joystick,  and  another  is  placed  on  the 
actuator  to  be  measured.  Rate  gyros  were  chosen 
since  both  the  joystick  and  the  camera’s  pan  actuator 
rotate  about  a  fixed  position.  Both  sensors  are 
connected  to  an  oscilloscope  and  the  latency 
measurement  can  be  measured  between  the  trigger 
signal,  e.g.  the  gyro  attached  to  the  joystick,  and  the 
resultant  output  gyro  attached  to  the  actuator. 

Two  things  need  to  be  taken  into  account  when 
measuring  this  latency.  The  first  is  that  a  joystick  has 
a  dead-zone  where  initial  movement  from  the  center 
position  doesn’t  generate  a  signal.  Measurement  of 
the  particular  joystick  being  used  should  lead  to  a 
latency  introduced  from  this  which  is  added  to  the 
overall  latency  equation.  The  second  is  the  response 
times  of  the  sensors.  In  this  case  there  are  two  rate 
gyros  so  the  overall  equation  has  a  ±2(/s,ro  )  term, 
which  leads  to  the  following  equation: 

^ system  —  ( joystick-deadzone  T  t OCU  T  t network 
T  l robot-cpu  +  ^robol -actuator 
—  ^  (Jgyro-response-time) 

The  latency  introduced  by  the  joystick’s  dead-zone  is 
part  of  the  system-under-test.  The  sensor  response 
times  are  an  artifact  of  the  measurement  system,  and 
need  to  be  taken  into  account  as  a  tolerance  on  the 
measurement,  but  are  not  part  of  the  latency  being 
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measured.  These  response  times  indicate  an  error 
plus  or  minus  for  the  overall  measurement. 

Video  Recording  ->  Display  on  an  OCU 

The  final  latency  test  measured  the  latency 
between  when  a  camera  records  an  image  to  when  it 
is  displayed  on  the  OCU.  For  this,  the  camera  is 
placed  a  dark  environment  (i.e.  light-tight  box)  with  a 
light  source,  and  an  image/pattem  to  record.  The 
light  source  is  controlled  via  a  switch  which  is  also 
connected  to  the  oscilloscope.  When  the  light  source 
is  active  the  oscilloscope  will  register  the  same 
voltage  as  the  light  source  is  using,  this  is  used  as  the 
first  pulse  for  the  latency  measurement.  When  the 
light  source  turns  on  the  camera  is  already  streaming 
the  video  feed,  in  this  case  a  dark  room/all  black 
image,  to  the  OCU  screen,  across  the  network.  Once 
the  light  source  is  turned  on  the  camera  has  to  record 
a  test  pattern,  compress  it,  and  transmit.  A  photo¬ 
detector  in  front  of  the  OCU  screen  captures  when 
the  image  has  changed  from  dark  to  a  lighted  test 
pattern. 


Electro-Mechanical  Non-lntrusive  Latency  Measurement  -  Case  3 


Figure  6:  Video  Latency 

This  gives  a  worst  case  latency  for  the  time  it 
takes  to  compress  a  new  image,  transmit  it,  and 
display  it  on  the  OCU  screen. 
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Data  Collection  and  Processing 


Figure  Figure  7  shows  a  typical  sample  from  the 
latency  test  involving  two  rate  gyros.  The  top  data 
view  shows  the  signals  in  their  raw  captured  form, 
and  shows  a  60  Hz  noise  introduced  from  the 
environment.  The  bottom  data  view  shows  the 
signals  after  being  run  through  a  60Hz  filter. 


Figure  7:  Rate  Gyro  Test  Capture 

The  signal  color-coded  white,  is  the  voltage  output 
from  the  rate  gyro  attached  to  a  joystick.  The  pulse 
indicates  that  the  joystick  has  moved  forward  (e.g. 
rising  edge)  and  then  moves  backward  (e.g.  falling 
edge/joystick  returning  to  center  position).  The 
ringing  at  the  end  of  the  falling  edge  is  the  joystick 
bouncing  around  the  center  position.  The  red  signal 
is  the  output  from  the  rate  gyro  attached  to  the  top  of 
the  PTZ  camera,  depicted  in  Figure  .  Since  the 
camera  turns  and  stops,  there  is  no  falling  edge  on 
this  signal.  After  filtering  the  signals  in  a  software 
program  that  comes  with  the  oscilloscope  used  for 
capturing  and  processing  signals  obtained  by  the 
oscilloscope,  the  signals  are  exported  to  ASCII  files 
for  processing. 


Rate  Gyro  positioned  to 
Measure  panningof  PTZ 


Figure  8 
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A  simple  program  was  developed  to  measure  the 
steady  state  response  average  voltage  for  50  ms 
before  the  rising  edge  of  the  curves,  and  for  another 
50  ms  after  the  rising  edge.  Figure  9  shows  a  latency 
measure  from  roughly  75%  of  the  rise  time  of  the  two 
pulses.  The  latency  is  derived  from  subtracting  the 
time  at  which  each  of  these  points  is  sampled. 


Figure  9:  Latency  Measure 

Choosing  Measurement  Points 

Both  precision  and  accuracy  of  the  measurement 
methods  have  to  be  characterized  in  order  for  these 
methods  to  be  utilized  against  real  systems.  Since  a 
system  with  independently  verified  latency  was  not 
available,  the  accuracy  of  the  demonstrated  method 
could  not  be  tested.  This  does  not  mean  that  the 
measurement  method  cannot  be  developed.  It  simply 
means  that  the  accuracy  of  the  demonstrated 
measurement  technique  cannot  be  defined  with 
confidence. 

In  order  to  process  the  data  from  a  measurement 
test,  two  points  have  to  be  chosen,  one  from  each  of 
the  signals  captured.  These  points  also  need  to  be 
chosen  avoid  noisy  areas.  Thus,  the  simple  logic  of 
measuring  from  close  to  the  start  of  output  signal 
rising  edge  does  not  always  work  since  the  there 
could  be  noise  in  near  the  start  of  the  rising  edge  as 
can  be  seen  in  Figure  9.  Although  this  could  be 
manually  judged  and  an  appropriate  point  determined 
by  “eye  balling”,  ideally  a  more  automated  method 
would  be  devised  to  determine  the  measurement 
points  so  that  the  measurements  can  be  consistent 
between  test  iterations.  Choosing  these  points  is 
problematic  when  the  two  sensor’s  responses  differ. 


For  instance,  referring  to  Figure  9,  it  can  be  seen  that 
the  two  rate  gyros  are  measuring  two  different  rates 
of  rotation.  Thus  the  differing  slopes  on  the  rising 
edge  of  the  signals.  Figure  10  is  a  simplification  of 
the  signals  utilizing  straight  lines. 

It  can  be  seen  from  this  figure  that  the  output  of 
the  rate  gyro  on  the  controller  (joystick),  has  a  faster 
slope  than  the  output  of  the  rate  gyro  on  the  platform 
actuator.  This  is  due  to  how  fast  the  rate  gyros  record 
the  events.  In  this  case  the  human  running  the  test 
has  moved  the  joystick  faster  than  the  actuator  on  the 
camera  will  move  the  camera. 


Figure  10:  Differing  Rise  Time  Rates:  Slower  Actuation 

Notice  that  since  the  second  signal  has  a  slower 
rate,  if  the  latency  was  determined  using  the  50%  rise 
time  points,  the  measured  time  would  be  greater  than 
the  actual  time  the  actuator  is  responding  to  that 
command.  In  other  words,  the  recorded  times  have 
spread  out.  This  indicates  that  if  the  latency  was 
measured  purely  by  using  the  50%  rise  time  of  both 
signals,  and  the  rate  of  the  actuator  was  slower  than 
the  rate  of  the  joystick,  the  resultant  latency  would  be 
greater  than  the  actual  latency. 

Should  the  slope  of  the  input  signal  be  slower 
than  the  output  signal,  the  opposite  happens,  as 
shown  in  Figure  1 1 .  Here  the  accuracy  of  the  latency 
measurement  has  converged  on  the  actual  value,  but 
still  isn’t  accurate.  In  this  case,  a  latency  measured 
would  be  smaller  than  the  actual  latency  of  the 
system.  Ideally,  the  slope  of  the  rise  times  would  be 
the  same  for  both  signals  in  order  to  prevent  either  of 
these  circumstances,  but  using  different  types  of 
sensors,  or  even  the  same  type  of  sensors  measuring  a 
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rate  of  change  that  differs  from  each  other,  lead  to 
these  situations. 
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Figure  11:  Differing  Rise  Time  Rates:  Faster  Actuation 


Example  Results  From  PTZ 

One  of  the  experiments  accomplished  was  to  use 
the  second  latency  test  ( command  actuation)  on 
the  PTZ  with  its  proprietary  controller.  The 
proprietary  controller  consisted  of  a  display,  with  a 
joystick  for  movement  and  zoom,  with  a  serial 
interface  to  the  PTZ  camera.  This  served  two 
purposes.  The  first  was  to  show  that  multiple  types 
of  OCU  joysticks  could  be  tested,  and  the  second, 
was  to  show  differences  between  a  proprietary 
controller  with  a  serial  interface,  versus  the  SSMI 
OCU  and  JAUS  messaging.  Over  a  run  of  ten  tests 
the  proprietary  controller  had  an  average  of  324  ms 
latency  between  command  and  action  with  a  standard 
deviation  of  2.25  ms.  The  same  test  was  run  with  the 
SSMI  OCU  and  included  JAUS  messaging  over  a 
network  and  the  results  were  295  ms  latency  on 
average,  with  a  standard  deviation  of  1 .92  ms.  There 
was  a  problem  with  the  data  capture  of  the  second 
test  set  where  the  captured  signals  did  not  have  data 
samples  to  get  a  good  average  of  the  steady  state  of 
the  actuator  mounted  gyro  after  the  rise  of  the  signal. 
This  is  due  to  the  set  up  of  the  oscilloscope,  and  can 
be  corrected  easily  in  the  future. 


Future  Vision 

The  VSIL  effort  has  shown  the  capability  to 
provide  validation  for  the  initial  IOP  effort  using 
JAUS  messages.  The  goal  is  to  evolve  the  IOP  over 
time  using  a  Working  Integrated  Product  Team 
(WIPT)  structure.  It  is  envisioned  that  the  VSIL  will 
continue  to  evaluate  the  message  set  as  it  evolves. 

The  architecture  approach  used  during  this  initial 
phase  was  fairly  straight  forward  and  the  proposed 
messages  achieved  the  desired  effects.  In,  future  IOP 
versions  this  may  not  be  the  case,  approaches  may 
not  end  with  desired  results  or  different  approaches 
may  need  to  be  evaluated  for  latency  and  other 
operational  parameters.  The  VSIL  in  this  instance 
may  be  needed  to  perform  a  testing  method  for  these 
different  approaches. 

As  the  Army’s  use  of  robotics  evolves,  especially 
into  more  autonomous  missions  and  roles  the  needs 
for  more  and  varied  architecture  approaches  to  the 
IOP  are  going  to  be  required.  These  approaches  and 
the  resulting  operational  results  will  be  difficult  to 
quantify  for  testers.  This  will  require  many  more 
iterations  of  the  same  test  as  autonomy  does  not 
always  provide  the  same  answer.  These  iterations 
will  provide  for  a  ‘bounding’  of  the  solution  set.  This 
‘bounding’  set  will  need  to  be  determined  by  testers 
to  be  sufficient  or  not.  Since,  traditional  range  testing 
is  costly  and  relatively  slow,  much  of  this  testing  will 
need  to  be  done  in  a  lab  setting,  using  M&S  as 
stimulation  and  in  some  cases  emulation  for  robotics 
systems  and  subsystems.  These  M&S  developed 
‘bounding’  sets  for  test  solutions  will  still  be 
validated  with  ‘live’  traditional  range  testing  to 
confirm  that  they  perform  within  the  ‘bounding’  sets. 
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